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1.  Introduction 


Detection  of  people  is  one  of  the  important  tasks  in  Intelligence,  Surveillance,  and 
Reconnaissance  (ISR)  missions.  For  example,  for  perimeter  protection,  one  would  like  to  detect 
intruders  both  during  day  and  at  night,  so  security  can  be  alerted  for  appropriate  action.  In  urban 
operations,  one  would  like  to  ensure  that  once  a  building  is  evacuated,  no  one  re-enters  the 
building — this  implies  that  sensors  need  to  detect  people  entering  the  building.  The  Department 
of  Homeland  Security  has  a  requirement  to  detect  illegal  immigrants  crossing  the  border.  There 
are  numerous  other  applications  where  personnel  detection  is  important. 

Detecting  people  is  a  challenging  problem.  For  example,  acoustic  sensors  may  analyze  sound  to 
determine  if  human  voices  are  present;  however,  if  people  are  not  talking,  the  acoustic  sensors 
will  not  be  able  to  detect  people  based  on  voice  analysis  alone.  Other  sensors  may  be  combined 
with  acoustic  sensors  such  as  seismic,  non-imaging  passive  infrared  (PIR),  sonar,  ultrasound, 
radar,  magnetic,  and  electric  field  sensors.  Multiple  sensing  modalities  are  beneficial  since  no 
single  sensor  can  detect  the  presence  of  humans  in  every  situation  and  circumstance.  There  is  an 
emphasis  on  non- imaging  sensors  because  they  require  lower  power,  last  longer,  and  generally 
are  less  expensive  than  imaging  sensors.  Imaging  sensors  require  higher  power  and  frequent 
replacement  of  batteries — hence,  there  is  a  higher  chance  of  compromising  the  mission. 

Traditionally,  personnel  detection  research  concentrated  on  footstep  analysis  using  seismic 
sensors.  When  a  person  walks,  the  person’s  foot  impacts  the  ground  causing  vibrations,  which 
are  detected  by  the  seismic  sensors.  In  general,  a  person  walks  with  a  rhythm  and  has  a  cadence 
frequency  of  approximately  1.5  Hz.  One  objective  of  the  research  was  to  detect  and  estimate  this 
cadence  frequency  to  distinguish  between  a  person  and  other  targets,  such  as  animals.  However, 
when  there  are  several  people  walking  in  a  group,  it  is  difficult  to  estimate  the  cadence  of  an 
individual  person  and,  in  fact,  it  is  quite  difficult  to  distinguish  between  people  and  animals 
walking.  In  order  to  aid  in  differentiating  people  from  animals,  a  multimodal  sensor  suite  was 
used.  Each  sensor  has  unique  characteristics  that  complement  other  sensing  modalities  in 
conducting  ISR;  section  2  provides  a  detailed  description  of  the  sensors  used  in  this  experiment. 


2.  Sensor  Suite 


2.1  Acoustic 

The  acoustic  sensors  used  for  this  experiment  were  piezoelectric  microphones.  Acoustic  sensors 
are  low-cost  sensors  used  to  detect  sound  energy  generated  by  people,  animals,  or  machines. 
Unlike  other  sensors,  these  sensors  are  omnidirectional  and  do  not  require  line  of  sight  to  a 
target.  If  more  than  one  microphone  is  employed,  these  acoustic  sensors  can  provide  a  line  of 


1 


bearing  to  the  source  of  the  sound.  Figure  1  shows  the  tetrahedral  array  of  microphones  used  in 
collecting  the  data.  The  microphones  used  for  this  experiment  were  three  Knowles  microphones 
(part  number  [P/N]  BL-1994)  with  a  frequency  response  of  20  Hz  to  20  kHz  placed  at  the  base  of 
the  tetrahedron  array,  and  a  Bruel  &  Kjaer  (B&K)  '/2-in-diameter  microphone  (P/N  4191)  with  a 
frequency  response  up  to  50  kHz  placed  on  the  z-axis  of  the  tetrahedron.  Each  B&K  microphone 
also  required  a  preamplifier  (P/N  2669)  and  power  supply  (P/N  5939L).  The  data  were  collected 
at  10  K  samples  per  second,  providing  an  effective  bandwidth  of  5  kHz,  which  allows  capture  of 
both  voice  and  (audible)  footstep  signatures  with  sufficient  fidelity  to  detect  personnel. 


Figure  1.  Single  microphone  and  an  array  (tetrahedral)  of 
microphones  with  windscreens. 


2.2  Seismic 

Seismic  sensors  are  similar  in  operation  to  acoustic  sensors  and  are  used  to  capture  vibrations  in 
the  ground  caused  by  footfalls  from  people  or  animals  walking,  or  by  vehicles  traveling  in  the 
vicinity,  among  other  sources.  The  geophone  seismic  sensors  used  during  our  experiment  were 
manufactured  by  Geospace  (P/N  GS-3C)  and  they  measure  vibrations  in  all  three  axes.  They 
were  oriented  towards  magnetic  north  and  were  buried  about  1 5  cm  deep  to  increase  coupling 
with  the  seismic  signals  (figure  2  shows  a  photo  of  the  sensor).  The  sensitivity  and  frequency 
response  of  seismic  sensors  generally  vary  depending  on  design  and  manufacturer  and  are 
typically  purchased  taking  mission  requirements  into  account. 
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Figure  2.  Tri-axis  seismic  sensor. 


2.3  Passive  Infrared  (PIR) 


These  non-imaging  devices  are  inexpensive  sensors  that  detect  the  nearby  presence  of  a  warm 
body,  e.g.,  a  human.  The  field  of  view  (FOV)  of  the  sensor  is  determined  by  the  lens  in  front  of 
the  actual  sensing  element.  Two  different  PIRs  were  employed,  one  with  an  FOV  of  15°  and  one 
with  60°.  In  addition,  the  PIR  sensors  have  plates  that  are  charged  differentially;  they  are  charged 
positively  or  negatively  depending  on  the  direction  the  person  is  traveling.  Hence,  it  is  possible 
to  determine  the  direction  of  motion  of  a  target.  The  sensors  reliably  detect  targets,  but  do  not 
distinguish  whether  the  target  is  human,  animal  or  a  warm  inanimate  object.  It  only  records  the 
thermal  signature  generated  by  the  target.  Figure  3  shows  a  photo  of  the  PIR  sensor  that  was 
fabricated  at  the  U.S.  Army  Research  Laboratory  (ARL)  and  used  for  the  experiment. 


Figure  3.  Picture  of  PIR  sensor. 


2.4  Magnetic  (B-fleld) 

These  sensors  can  be  used  to  detect  ferromagnetic  materials  carried  by  people,  e.g.,  keys, 
firearms,  and  knives.  For  the  experiment  we  used  Model  1540  three-axis  fluxgate  magnetometers 
from  Applied  Physics  Systems.  They  have  a  sensitivity  of  30  pV/nT  and  are  used  to  detect  low 
frequency  components  up  to  5  Hz.  The  data  from  the  fluxgate  magnetometer  were  collected  at  10 
samples  per  second.  Figure  4  shows  a  photo  of  the  emplaced  fluxgate  magnetometer. 
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Figure  4.  Picture  of  fluxgate  magnetometer. 


2.5  Electrostatic 

The  sensor  used  for  the  experiment  was  an  ARL-built  E- field  sensor.  The  sensor  consists  of  two 
parallel  plates  with  the  lower  plate  grounded  with  a  spike  driven  into  the  ground.  The  plates  are 
charged  and  when  a  person  having  a  static  E-field  (due  to  rubbing  of  clothes,  etc.)  walks  nearby 
the  plate  potential  is  affected.  The  sensor  output  was  recorded  at  10  K  samples  per  second. 
Figure  5  shows  a  photo  of  an  emplaced  E-field  sensor. 


Figure  5.  Picture  of  E-field  sensor. 


2.6  Modified  Toy  Radar 

This  is  a  stock  Mattel  Hot  Wheels  Radar  Gun  with  a  transmit  frequency  of  10.525  GHz  that 
displays  the  velocity  of  the  target  in  km/hr  (or  mi/hr)  that  was  modified  to  provide  the  Doppler 
frequency  of  the  target  (fd=2v/A,).  The  toy  radar  used  during  the  data  collection  is  shown  in 
figure  6. 
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Figure  6.  Mattel  Flot  Wheels  radar  gun. 


2.7  ARL  Compact  Radar 

The  ARL  compact  radar  is  a  Doppler-based,  linear  frequency  modulated,  low-power  radar 
system  with  an  electrically  scanned  patch-array  antenna  that  allows  azimuth  and  elevation 
control  of  the  radar  beam.  The  radar  system  provides  sufficient  resolution  to  detect  movement  of 
individual  arms  and  legs  on  humans  or  animals  walking  or  running — the  limb  movement 
velocity  information  is  used  to  differentiate  between  humans  and  animals.  Figure  7  shows  a 
photo  of  the  ARL  compact  radar. 


Figure  7.  ARL  compact  radar. 


2.8  Ultrasonic 

This  sensor  transmits  an  acoustic  tone  at  40  kHz  and  receives  the  signals  scattered  by  the  target, 
both  active  and  passive  ultrasonic  measurements  were  conducted.  For  the  passive  method,  a 
narrowband  ultrasonic  microphone  receiver  (UR)  was  used  to  measure  ultrasonic  emissions  from 
footsteps.  This  UR  had  a  resonance  frequency  of  25.5  kHz  and  a  bandwidth  of  1  kHz  at  -6  dB, 
the  directivity  at  -6  dB  was  60°.  For  the  active  method  a  40-kHz  ultrasonic  continuous- wave 
(CW)  Doppler  sensor  (UDS)  was  employed.  The  sensor  was  assembled  from  two  40-kHz 
ultrasonic  transducers  (MATSU/PAN  EFR-RCB40K  54).  These  transducers  have  a  typical 
bandwidth  at  -6  dB  of  2  kHz  and  a  directivity  pattern  at  -6  dB  of  55°.  One  of  the  transducers, 
acting  as  a  transmitter,  emitted  an  ultrasonic  wave,  while  the  other  acted  as  a  receiver.  Signals 
from  the  UR  and  the  UDS  were  simultaneously  recorded  using  a  12-channel,  24-bit  at  1  V  data 
acquisition  board  (DAQ)  (Echo  AudioFirel2)  and  a  laptop  computer  with  Loud  Technologies 
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software  (Tracktion  2).  The  DAQ,  having  a  sampling  rate  of  96  kHz,  acquired  signals  from  the 
sensors.  These  signals  were  stored  on  the  laptop  computer  in  a  .wav  format.  The  UR  and  UDS 
were  attached  to  a  1.2-m  tall  tripod.  In  the  test  configurations,  the  beam  pattern  of  the  UR  and 
UDS  were  oriented  along  the  walking  track.  Figure  8  shows  the  ultrasonic  sensor  used. 


Figure  8.  Ultrasonic  sensor. 


2.9  Profiling 

This  sensor  records  the  silhouette  of  the  target  rather  than  the  image  of  the  target.  One 
implementation  of  the  profiling  sensor  is  shown  in  figure  9a  with  two  vertical  poles  consisting  of 
IR  transmitters  mounted  on  one  pole  and  the  receivers  mounted  on  the  other.  A  sample  human 
silhouette  is  shown  in  figure  9b.  A  more  covert  type  of  profile  sensor  built  by  Brimrose  that  can 
be  emplaced  some  distance  from  the  trail  is  shown  in  figure  9c. 


(c) 

Figure  9.  (a)  Two-pole  profiling  sensor,  (b)  typical  human  silhouette,  and  (c)  Brimrose 
profiling  sensor. 
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2.10  Visible  Imaging 

Each  ARL  sensor  suite  also  included  a  Kodak  Zi8  video  color  camera  to  record  ground  truth  for 
each  data  run;  the  video/audio  data  were  stored  on  32-GB  SanDisk  memory  cards.  The  video 
camera  has  a  resolution  of  1920x1080  pixels  at  30  frames  per  second  using  a  6.3-mm  f/2.8  fixed 
focus  lens.  The  camera  also  records  audio.  Figure  10  shows  a  picture  of  the  visible  video  camera 
set  up  along  the  trail  and  a  close-up  view. 


Figure  10.  Picture  of  daylight  video  camera  set  up  along 
trail  and  close  up. 


2.11  Visible  and  Infrared  Imaging 

These  imagers  are  commonly  used  in  the  ISR  realm  based  on  the  adages  that  a  “picture  is  worth 
thousand  words”  or  “seeing  is  believing.”  The  dual  imager  used  for  this  experiment  came  from  a 
Tactical  Remote  Sensor  System  (TRSS)  and  it  contains  both  a  visible  and  IR  camera.  The  visible 
color  video  camera  has  a  resolution  of  768x494  active  pixels,  14-in  charge-coupled  device 
(CCD),  and  has  approximately  450  TV  lines.  The  FOV  is  approximately  25°.  The  IR  camera 
used  is  an  uncooled  long-wave  (8-12  micron)  microbolometer  with  FOV  of  25°  and  a  focal 
plane  array  resolution  of  320x240.  The  data  were  collected  at  30  frames  per  second  and  recorded 
using  an  ARL  integrated  four-channel  digital  video  recorder  consisting  of  four  Datavideo 
120-Mbyte  hard  drives  and  associated  control  circuitry.  Figure  1 1  shows  the  emplaced  TRSS 
dual  imager  and  ARL  recording  equipment  used  for  the  experiment,  along  with  a  close  up  of  the 
TRSS  imager. 
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Figure  11.  Picture  of  emplaced  TRSS  dual  imager  and  ARL  data  recorder,  plus  a  close-up  shot  of  imager. 

2.12  Meteorological  Station 

The  air  temperature,  wind  speed  and  direction,  dew  point,  barometric  pressure,  and  humidity 
along  with  a  global  positioning  system  (GPS)  time  stamp  were  recorded  once  every  minute  using 
a  Vantage  Pro  2  Weather  Station  from  Davis  Instruments  for  the  duration  of  the  experiment  at 
each  test  site.  At  the  end  of  each  day,  the  meteorological  data  were  retrieved  from  the  Vantage 
Pro  Remote  Controller  and  downloaded  onto  a  laptop  and  stored  in  a  text  file.  Figure  12  shows  a 
picture  of  an  emplaced  Vantage  Pro  2  Weather  Station. 


Figure  12.  Picture  of  Vantage  Pro  2  weather  station. 
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3.  Data  Collection 


In  order  to  collect  data  in  a  real-world  situation,  the  ARL  team  went  to  the  U.S.  southwest 
border.  Two  trips  were  made:  the  first  visit  was  to  consult  with  the  Department  of  Homeland 
Security,  which  faces  the  illegal  immigration  problem  on  a  daily  basis,  and  determine  the  actual 
test  sites  where  the  experiments  would  be  conducted.  After  discussions  with  the  border  security 
division  and  watching  surveillance  videos,  it  was  determined  the  experiment  would  occur  at 
three  different  types  of  locations,  namely  a  wash,  a  trail,  and  a  choke  point.  Figures  13a-c  show 
pictures  of  a  wash,  trail,  and  choke  point  (respectively)  that  are  typically  found  along  the 
southwest  border. 


(c) 


Figure  13.  (a)  Photo  of  a  wash,  (b)  a  trail,  and  (c)  a  choke  point. 
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3.1  Data  Collection  Objectives 

Several  objectives  were  envisioned  for  the  field  test  experiment  on  the  southwest  border,  these 
included  the  collection  of  personnel,  animal,  and  vehicle  signatures  using  an  array  of  sensors  for 
the  purpose  of  understanding  target  phenomenology  and  providing  signatures  for  accomplishing 
the  following: 

•  Developing  multi-sensor  fusion  algorithms 

•  Developing  detection  and  classification  algorithms  for  single-  and  multi-sensor  systems 

•  Supporting  target  counting  and  direction  of  travel  algorithms 

•  Supporting  algorithm  development  to  determine  if  payloads  are  being  carried  by  humans  or 
animals 

3.2  Participating  Teams 

The  following  lists  the  teams  that  participated  in  the  experiment  and  what  tasks  they  handled: 

•  ARL 

-  Networked  Sensing  and  Fusion  Branch 

•  Acoustics,  seismic,  PIR,  E  and  B-field,  and  digital  video  (truth  data) 

-  Radar  Branch 

•  Compact  radar 

•  Armament  Research  and  Development  Center  (ARDEC) 

-  Test  support 

•  University  of  Mississippi 

-  Acoustic,  seismic,  active  and  passive  ultrasonic,  and  toy  radar 

•  University  of  Memphis 

-  Profiling  sensors  (parallel  optical  tripwire  and  microbolometer) 

•  Brimrose  Co. 

-  Thermopile  profiling  sensor 

•  Night  Vision  and  Electronic  Sensors  Directorate  (NVESD) 

-  Pyroelectric  profiling  sensor 
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3.3  Sensor  Placement 


The  experiment  used  four  ARL  sensor  suites  placed  approximately  50  m  apart  along  the 
prescribed  travel  path.  Each  sensor  suite  consisted  of  acoustic,  seismic,  PIR,  magnetic,  E-field, 
and  toy  radar  sensors.  Figure  14  shows  a  typical  deployment  of  the  entire  sensor  ensemble  at  a 
test  site  and  figure  15  shows  a  photo  of  an  individual  ARL  sensor  suite.  The  ARL  compact  radar 
was  placed  at  a  location  in-line  with  the  trail  that  allowed  good  coverage  of  the  subject 
movement  along  the  entire  travel  path,  while  the  TRSS  visible  and  infrared  imager  was  placed  at 
a  vantage  point  off  to  one  side  of  the  trail. 


TRSS 

FLIR 


ARL  Compact 
Radar  '  A 


Memphis  Profile 
Sensor  (POT) 


100m 

ARL2  Sensor  suite 
with  Compact  RIO 

50m 


<— 


50m 


ARL4  Sensor  suite 
with  Compact  RIO 


6* 


10m 


B  &  E  -  field  sensors 

im  y 

* — j-i  L-.  ARL1  Sensor  suite 
L|j-I  with  Compact  RIO 


— > 


50m 


50m 


10m 


ARL3  Sensor  suite 
with  Compact  RIO 

Brimrose  Profile 
Sensor 


Figure  14.  Typical  sensor  layout. 
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Figure  15.  Typical  ARL  sensor  suite. 

3.4  Data  Collection  Equipment 

Each  ARL  sensor  suit  used  a  12-channel,  16-bit  National  Instruments  CompactRIO  control  and 
data  acquisition  system  for  capturing  the  various  sensor  signals.  Major  advantages  of  this 
recording  system  include  low  power  consumption;  small  size  and  operation  from  12-V 
automotive  batteries  for  portability;  low  noise  operation  (no  cooling  fan)  that  produced  very  little 
acoustic  and  electrical  noise  that  might  otherwise  corrupt  sensor  signals;  and  the  inclusion  of  a 
GPS  receiver  that  accurately  time  stamped  all  the  recorded  data.  The  partitioning  of  the  12 
CompactRIO  channels  and  the  sampling  rate  for  the  sensors  are  shown  in  table  1 .  Each  ARL 
sensor  suite  also  included  a  Kodak  Zi8  daylight  video  camera  to  record  ground  truth  that  was 
stored  on  32-GB  SanDisk  memory  cards. 


Table  1.  Apportionment  of  CompactRIO  channels  and  sampling  rate. 


Sensor  Type 

No.  Channels 

Sampling  Rate 

Resolution 

Acoustic 

4  -  Analog 

10000 

16  bit 

Seismic 

3  -  Analog 

10000 

16  bit 

PIR 

1  -  Analog 

10000 

16  bit 

E-field 

1  -  Analog 

10000 

16  bit 

Toy  Radar 

1  -  Analog 

10000 

16  bit 

Magnetic 

1  -  Digital  (3  axis) 

10 

24  bit 
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3.5  Data  Collection  Scenarios 


The  following  data  collection  scenarios  were  used  for  the  experiment.  For  the  cases  where  a 
small  animal  was  used,  the  handler  walked  behind  the  animal.  For  the  large  animals,  the  handlers 
walked  in  front. 

Run  #1:  Pre-test  checkout  of  equipment  to  make  sure  sensors  and  recording  equipment  were 
operating  properly 

•  Acoustic  Sensors:  We  applied  a  1-kHz  signal  to  microphones  using  an  acoustic  source 
(B&K  P/N  4230). 

•  Seismic  Sensors  (oriented  magnetic  north):  Dropped  a  6.5-kg  steel  weight  from  1-m  height 
at  1,  5,  and  10  m  due  north  from  sensor. 

•  PIR  Sensors:  One  person  walked  through  FOV  of  sensor  at  1,  5,  and  10  m  from  sensor. 

•  Magnetic  Field  Sensors:  A  person  walked  past  the  sensor  with  a  1-m  ferrous  metal  pipe 
(5  cm  diameter)  at  1,  5,  and  10  m  from  sensor. 

•  E-field  Sensors:  We  used  a  piezoelectric  sparker  (from  a  butane  lighter)  at  1,  5,  and  10  m 
from  sensor. 

•  ARL  compact  Radar:  One  person  walked  from  1  to  30  m  away  from  the  radar  at  constant 
velocity  and  then  back  towards  the  radar. 

Run  #2:  One  man  walking  (figure  16) 

•  We  noted  the  GPS  time  at  the  beginning  and  end  of  the  data  run. 

•  We  collected  background  ambient  data  for  2  min. 

•  A  person  walked  from  beginning  to  end  of  designated  path,  waited  10  s,  then  returned 
along  same  path  and  stood  still  for  another  10  s  when  finished. 

•  While  walking,  the  person  was  instructed  to  talk,  sing,  hum,  etc. 


Figure  16.  Run  #2. 
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Figures  17  through  20  illustrate  Runs  #3  though  #26.  Runs  #3  though  #26  followed  the  same 
data  collection  procedures  as  Run  #2;  they  only  varied  in  the  participants  and  activities 
performed. 


Figure  17.  (a)  Run  #3:  one  man  walking  with  a  40  lb  backpack,  (b)  Run  #4:  one  man  running,  (c)  Run  #5:  one 
woman  walking,  (d)  Run  #6:  one  woman  walking  with  20-30  lb  backpack,  (e)  Run  #7:  one  man  and 
one  woman  walking,  and  (f)  Run  #8:  two  women  walking. 
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(e)  (f) 

Figure  18.  (a)  Run  #9:  two  men  running,  (b)  Run  #10:  two  men  running  with  backpacks,  (c)  Run  #11:  three 


men  walking,  (d)  Run  #12:  two  men  and  one  woman  walking,  (e)  Run  #13:  three  men  running,  and 
(f)  Run  #14:  two  men  and  one  woman  running. 
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Figure  19.  (a)  Run  #15:  three  men  running  with  backpacks,  (b)  Run  #16:  four  men  and  three  women  walking, 

(c)  Run  #17:  one  dog  with  one  woman  walking,  (d)  Run  #18:  one  horse  with  handler  walking,  (e)  Run 
#19:  one  horse  and  one  mule  with  handlers  walking,  and  (f)  Run  #20:  one  horse,  one  mule,  and  one 
donkey  with  three  handlers  walking. 
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Figure  20.  (a)  Run  #21 :  one  horse,  one  mule,  and  one  donkey  with  three  handlers  walking,  (b)  Run  #22:  one  horse 
with  payload  with  handler  walking,  (c)  Run  #23 :  one  horse  and  one  mule  with  payloads  with  two 
handlers  walking,  (d)  Run  #24:  one  horse,  one  mule,  and  one  donkey  with  payloads  with  three  handlers 
walking,  (e)  Run  #25:  one  ATV  going  5  mph,  and  (f)  Run  #26:  another  ATV  going  10  mph. 
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Tables  2-7  provide  information  on  sensor  locations  and  test  schedules.  All  times  listed  are  local 
mountain  standard  time  (MST);  individual  test  subjects  abbreviated  as  Ml  (man  1),  M2  (man  2), 
W1  (woman  1),  etc. 

Table  2.  Sensor  locations  for  test  site  2,  Tuesday,  December  8, 2009. 


Sensor  Type 

Elevation 

(ft) 

Waypoint 

Name 

Site 
Photo 
(Y  or  N) 

Notes 

MET  Station 

4423 

Wpt  002 

y 

Compact  RIO  #2A 

4417 

Wpt  003 

y 

Site  also  contained  radar  gun  and  digital 
video  camera 

Compact  RIO  #3  A 

4416 

Wpt  004 

y 

Site  also  contained  radar  gun  and  digital 
video  camera 

Compact  RIO  #5A 

4440 

Wpt  005 

y 

Site  also  contained  radar  gun 

TRSS 

4459 

Wpt  006 

y 

ARL  Compact  Radar 

4443 

Wpt  009 

y 

Radar  mounted  on  top  of  vehicle 

Brimrose  PFx  and 

Memphis  FLIR 

4447 

Wpt  008 

y 

Memphis  PFx 

4437 

Wpt  007 

y 

Table  3.  Test  scenarios  and  data  collection  start  and  end  times  for  test  site  2,  Tuesday,  December  8,  2009. 


Run 

No. 

Test  Scenario 

Start  time 

Stop  time 

Notes 

2 

One  man  walking 

11:52:00  AM 

Ml 

3 

One  man  walking  with  40-lb  backpack 

12:00:00  Noon 

12:05:00  PM 

M2 

4 

One  man  running 

12:07:00  PM 

12:10:00  PM 

M2 

5 

One  woman  walking 

12:11:00  PM 

12:18:27  PM 

W1 

6 

One  woman  walking  with  20-lb 
backpack 

12:19:49  PM 

12:18:27  PM 

W2 

7 

One  man  and  one  woman  walking 

12:26:15  PM 

M3,  W3 

8 

Two  women  walking 

Wl,  W2 

9 

Two  men  running 

12:47:24  PM 

12:52:21  PM 

M4,  M2 

10 

Two  men  running  with  40-lb  backpacks 

12:54:00  PM 

M2,  M3 

11 

Three  men  walking 

1:00:10  PM 

1:06:38  PM 

M3,  M4,  Ml  1 

12 

Two  men  and  one  woman  walking 

1:08:01  PM 

1:14:54  PM 

M4  with  large  iron  pipe, 

Ml  1,  Wl 

13 

Three  men  running 

1:16:04  PM 

1:20:35  PM 

M2,  M5,  M3 

14 

Two  men  and  one  woman  running 

1:21:43  PM 

1:27:20  PM 

Ml 3,  M2,  W3 

15 

Three  men  running  with  40  lb  backpacks 

1:43:10  PM 

1:48:48  PM 

M2,  Ml,  M4,  Compact 

RIO  2A  stopped  working 

16 

Four  men,  three  women,  and  one  dog 
walking 

1:54:48  PM 

2:02:04  PM 

Ml,  M2,  M4,  M6,  Wl,  W2 
with  dog,  W3.  Stopped 
test  for  lunch  break. 

17 

One  dog  with  one  woman  walking 

2:50:00  PM 

2:55:01  PM 

W2 

22a 

Man  leading  horse  with  payload 

2:56:00  PM 

3:02:04  PM 

M7 

23a 

Two  men  leading  horse  and  mule  with 
payloads 

3:04:11  PM 

3:10:08  PM 

M7,  M8 

24a 

Three  men  leading  horse,  mule,  and 
donkey  with  payloads 

3:11:00  PM 

3:17:00  PM 

M7,  M8,  M9 
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Table  3.  Test  scenarios  and  data  collection  start  and  end  times  for  test  site  2,  Tuesday,  December  8,  2009 
(continued). 


Run 

No. 

Test  Scenario 

Start  time 

Stop  time 

Notes 

21a 

Three  men  walking  beside  horse,  mule, 
and  donkey  with  payloads 

3:17:55  PM 

3:24:24  PM 

M7,  M8,  M9 

18 

Man  leading  horse 

3:36:50  PM 

3:42:26  PM 

M7 

19 

Two  men  leading  horse  and  mule 

3:43:52  PM 

3:49:58  PM 

M7,  M8 

20 

Three  men  leading  horse,  mule,  and 
donkey 

3:50:18  PM 

3:55  PM 

M7,  M8,  M9 

22 

Man  leading  horse  with  saddle  pack, 

100  lb  load 

4:00:00  PM 

4:04:51  PM 

M7 

23 

Two  men  leading  horse  and  mule  with 
saddle  packs,  100  lb  loads 

4:05:23  PM 

4:11:00  PM 

M7,  M8 

24 

Three  men  leading  horse,  mule,  and 
donkey  with  saddle  packs,  1 00  lb  loads 

4:11:19  PM 

4:17:10  PM 

M7,  M8,  M9 

Table  4.  Sensor  locations  for  test  site  2,  Wednesday,  December  9,  2009. 


Sensor  Type 

Elevation 

(fo 

Waypoint 

Name 

Site 
Photo 
(Y  or  N) 

Notes 

MET  station 

4423 

Wpt  002 

y 

Same  location  as  day  2 

Compact  RIO  #2A 

4447 

WptOlO 

y 

Site  also  contained  radar  gun  and 
digital  video  camera 

Compact  RIO  #3  A 

4445 

Wpt  011 

y 

Site  also  contained  radar  gun 

Compact  RIO  #5A 

4439 

Wpt  012 

y 

Site  also  contained  radar  gun 

TRSS 

4459 

Wpt  006 

y 

Same  location  as  day  2 

ARL  compact  radar 

4437 

Wpt  013 

y 

Radar  pointed  due  west 

Brimrose  PFx  and  Memphis  FLIR 

4438 

Wpt  017 

y 

Site  also  contained  FFIR  and  two 
linear  array  pyro  PIR’s,  all  pointed 
340° 

Memphis  PFx 

4437 

Wpt  007 

y 

Same  location  as  day  2 

Digital  movie  camera 

4447 

Wpt  014 

y 

Two  digital  movie  cameras  used 
during  the  test 

Mississippi  ultrasonic  and  radar 

4413 

Wpt  015 

y 

Black  body  radiator 

4438 

Wpt  018 

y 

Used  to  calibrate  FFIR  sensors 

Table  5.  Test  scenarios  and  data  collection  start  and  end  times  for  test  site  2,  Wednesday,  December  9,  2009. 


Run 

No. 

Test  Scenario 

Start  time 

Stop  time 

Notes 

25 

1  ATV  traveling  -5  mph 

12:15:45  PM 

12:20:13  PM 

Yamaha  Grizzly  750  FI 

26 

1  ATV  traveling  -10  mph 

12:24:17  PM 

12:27:44  PM 

Yamaha  Grizzly  750  FI 

27 

1  ATV  traveling  -5  mph  with  man 
running  behind 

12:30:15  PM 

12:34:38  PM 

Yamaha  Grizzly  750  FI  and  M2 
running 

27a 

1  ATV  traveling  -5  mph  with  man 
running  behind 

12:38:17  PM 

12:42:27  PM 

Yamaha  Grizzly  750  FI  and  M2 
running 

28 

2  ATV’s  traveling  -10  mph 

12:46:15  PM 

12:49:52  PM 

2  Yamaha  Grizzly  750  FI 

29 

3  ATV’s  traveling  -10  mph 

12:52:55  PM 

12:57:54  PM 

2  Yamaha  Grizzly  750  FI  and  1 
Yamaha  Kodiak  Ultramatic  400 
(all  single-cylinder  engines) 
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Table  5.  Test  scenarios  and  data  collection  start  and  end  times  for  test  site  2,  Wednesday,  December  9,  2009 
(continued). 


Run 

No. 

Test  Scenario 

Start  time 

Stop  time 

Notes 

2 

One  man  walking 

1:02:05  PM 

1:08:29  PM 

Ml,  helicopter  noise 

3 

One  man  walking  with  40-lb  backpack 

1:11:10PM 

1:17:18  PM 

M2 

4 

One  man  running 

1:19:50  PM 

1:24:17  PM 

M2 

5 

One  woman  walking 

1:27:08  PM 

1:34:52  PM 

W1  (unsure  of  trail  path),  ATV 
started  during  ambient  data 
period,  added  another  minute, 
ATV  and  bird  noise  during  run. 

6 

One  woman  walking  with  20-lb 
backpack 

1:37:31  PM 

1:43:59  PM 

W2 

7 

One  man  and  one  woman  walking 

1:45:47  PM 

1:52:28  PM 

W3,  M3 

8 

Two  women  walking 

1:53:54  PM 

2:00:49  PM 

Wl,  W2 

9 

Two  men  running 

2:02:52  PM 

2:07:53  PM 

M4,  M2,  noise  from  Memphis 
gazebo  as  it  flapped  in  strong 
wind 

10 

Two  men  running  with  40-lb 
backpacks 

2:10:51  PM 

2:16:11  PM 

Ml 2,  M2,  helicopter  noise 
during  run 

11 

Three  men  walking 

2:18:25  PM 

2:25:04  PM 

M3,  M4,  Ml  1 

12 

Two  men  and  one  woman  walking 

2:26:24  PM 

2:32:50  PM 

M4  with  large  iron  pipe,  Mil, 

Wl 

13 

Three  men  running 

2:42:41  PM 

2:47:26  PM 

Ml 2,  M2,  M5,  ATV  noise  in 
background 

14 

Skipped  this  test  run 

15 

Three  men  running  with  40-lb 
backpacks 

2:51:00  PM 

2:56:13  PM 

M4,  M13,  M2.  Dog  barked  at 
end  of  run. 

16 

Four  men,  three  women,  and  one  dog 
walking 

2:57:25  PM 

3:04:29  PM 

Ml,  M2,  M4,  M6,  Wl,  W2  with 
dog,  W3 

17 

One  dog  with  one  woman  walking 

3:05:59  PM 

3:12:01  PM 

W2 

18 

Man  leading  horse 

3:17:09  PM 

3:23:18  PM 

M7,  Memphis  gazebo  making 
noise  flapping  in  wind 

19 

Two  men  leading  horse  and  mule 

3:24:16  PM 

3:30:19  PM 

M7,  M8,  ATV  hom  beeped 
3:26:10PM. 

20 

Three  men  leading  horse,  mule,  and 
donkey 

3:34:40  PM 

3:41:08  PM 

M7,  M8,  M9 

21 

Skipped  this  test  run 

22 

Man  leading  horse  with  saddle  pack, 

100  lb  load 

3:51:35  PM 

3:57:19  PM 

M7 

23 

Two  men  leading  horse  and  mule  with 
saddle  packs,  100-lb  loads 

3:59:00  PM 

4:04:56  PM 

M7,  M8,  cell  phone  rang 

3:59:10,  Memphis  gazebo 
flapping  noise  4:00:05,  walkie 
talkie  call  ringer  4:02:40,  jet 
passed  overhead  4:03 :3 1 . 

24 

Three  men  leading  horse,  mule,  and 
donkey  with  saddle  packs,  100-lb 
loads 

4:07:14  PM 

4:12:56  PM 

M7,  M8,  M9 

22a 

Man  leading  horse  with  payload 

4:37:00  PM 

M7 

23a 

Two  men  leading  horse  and  mule  with 
payloads 

4:38:00  PM 

M7,  M8 

24a 

Three  men  leading  horse,  mule,  and 
donkey  with  payloads 

4:40:00  PM 

M7,  M8,  M9 

22b 

Man  leading  donkey  with  payload 

4:42:00  PM 

M9 
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Table  6.  Sensor  locations  for  test  site  3,  Thursday,  December  10,  2009. 


Sensor  Type 

Elevation 

(m 

Waypoint 

name 

Site  Photo 
(Y  or  N) 

Notes 

MET  station 

4477 

Wpt  021 

y 

Compact  RIO  #2A 

4471 

Wpt  023 

y 

Compact  RIO  #3A 

4462 

Wpt  025 

y 

Compact  RIO  #5A 

4439 

Wpt  026 

y 

TRSS 

Did  not  set  up  TRSS  on  day  4 

ARL  compact  radar 

4440 

Wpt  028 

n 

Raju’s  camera  battery  died 

Brimrose  PFx  and  Memphis  FLIR 

4413 

Wpt  029 

n 

Memphis  PFx 

4477 

Wpt  022 

n 

Mississippi  ultrasonic  and  radar 

4455 

Wpt  024 

y 

Trail  start 

n 

GPS  receiver  battery  died 

Trail  end 

4433 

Wpt  027 

y 

Table  7.  Test  scenarios  and  data  collection  start  and  end  times  for  test  site  3,  Thursday,  December  10,  2009. 


Run 

No. 

Test  Scenario 

Start  time 

Stop  time 

Notes 

0 

Two  border  patrol  agents 

12:01:59  PM 

12:08:32  PM 

1  agent  with  Ml 6  rifle,  both 
with  sidearms  and  small 
backpacks 

18 

Man  leading  horse 

12:12:58  PM 

12:19:23  PM 

M7,  small  airplane  flew  over 
12:17:00 

19 

Two  men  leading  horse  and  mule 

12:22:28  PM 

12:28:46  PM 

M7,  M8,  red  tail  hawk  called 
from  above 

20 

Three  men  leading  horse,  mule,  and 
donkey 

12:31:45  PM 

12:38:12  PM 

M7,  M8,  M9 

21 

Skipped  this  test  run 

22 

Man  leading  horse  with  saddle  pack,  100- 
lb  load 

12:48:32  PM 

12:54:35  PM 

M7,  heard  ATV  and  walkie 
talkie  call  ringer 

23 

Two  men  leading  horse  and  mule  with 
saddle  packs,  100  lb  loads 

1:00:08  PM 

1:06:30  PM 

M7,  M8 

24 

Three  men  leading  horse,  mule,  and 
donkey  with  saddle  packs,  100-lb  loads 

1:08:59  PM 

1:15:20  PM 

M7,  M8,  M9 

22a 

Man  leading  horse  with  payload 

1:22:59  PM 

1:28:59  PM 

M7,  crow  called  in 
background 

23a 

Two  men  leading  horse  and  mule  with 
payloads 

1:30:52  PM 

1:37:33  PM 

M7,  M8 

24a 

Three  men  leading  horse,  mule,  and 
donkey  with  payloads 

1:41:40  PM 

1:47:45  PM 

M7,  M8,  M9 

2 

One  man  walking 

1:52:00  PM 

1:57:08  PM 

M10 

3 

One  man  walking  with  40-lb  backpack 

2:00:06  PM 

2:05:30  PM 

M4 

4 

Skipped  this  test  run 

5 

One  woman  walking 

2:07:18  PM 

2:13:22  PM 

W1 

6 

One  woman  walking  with  20-lb  backpack 

2:15:44  PM 

2:21:20  PM 

W2 

7 

One  man  and  one  woman  walking 

2:23:49  PM 

2:29:53  PM 

W3,M4 

8 

Two  women  walking 

2:43:43  PM 

2:49:50  PM 

Wl,  W2 

9 

Skipped  this  test  run 

10 

Skipped  this  test  run 

11 

Three  men  walking 

2:52:23  PM 

2:58:30PM 

Ml,  M4  with  large  iron  pipe, 
Mil 

21 


Table  7.  Test  scenarios  and  data  collection  start  and  end  times  for  test  site  3,  Thursday,  December  10,  2009 
(continued). 


Run 

No. 

Test  Scenario 

Start  time 

Stop  time 

Notes 

12 

Two  men  and  one  woman  walking 

3:00:42  PM 

3:06:54PM 

M4,  Ml  1,  W4,  jet  noise 
overhead  3:04:00 

13 

Skipped  this  test  run 

14 

Skipped  this  test  run 

15 

Skipped  this  test  run 

16 

Four  men,  three  women,  and  one  dog 
walking 

3:11:04  PM 

3:17:39  PM 

Ml,  Ml 2,  M4  with  large  iron 
pipe,  M6,  W4,  W3,  W2  with 
dog 

16a 

Background  ambient  noise  test 

3:20:49  PM 

3:24:00  PM 

Radio  talk  3:21:18,  voices  and 
radio  talking  3:22:15,  crow 
called  3:23:20 

4.  Data  Analysis  and  Algorithm  Development 


4.1  Acoustic  Sensor  Data  Analysis  for  Personnel  Detection 

The  analysis  of  the  acoustic  sensor  data  is  primarily  concentrated  on  three  aspects:  (1)  human 
voice  detection  by  way  of  formant  detection  (8),  (2)  spread  of  energy  of  human  voice  in  various 
spectral  bands  (7,  8),  and  (3)  footstep  detection  for  estimation  of  cadence. 

4.1.1  Detection  of  Formants 

Humans  generate  their  speech  sounds  by  modulating  the  vocal  cords  and  appropriately  opening 
and  closing  their  vocal  tract.  In  general,  there  are  several  frequencies  associated  with  voice  and 
they  are  called  formants.  A  small  segment  of  speech  signal  is  shown  in  figure  21.  One  would 
notice  from  figure  21  that  whenever  a  word  is  spoken  a  burst  of  high  frequency  signal  appears 
along  with  the  background  noise.  This  high  frequency  signal  is  called  the  formant  and  it  varies 
from  person  to  person  and  also  on  the  word  spoken.  In  general,  the  frequency  lies  between  200- 
800  Hz  for  the  people  we  tested.  Figure  22a  shows  an  expanded  version  of  the  first  segment  of 
the  voice  signal  shown  in  figure  21,  and  figure  22b  shows  the  Fourier  transform  of  the  segment. 
Clearly,  one  can  see  the  dominant  frequency  around  300  Hz.  The  objective  of  the  signal 
processing  is  to  detect  and  determine  this  frequency.  As  mentioned  previously,  the  carrier 
frequency  (formant)  is  amplitude  modulated  whose  representation  may  be  given  as 

HO  =  (A  +  Arn  sin(2^/mO) cos(2;r fct)  ( 1 ) 

where  “f”  and  represent  the  carrier  and  modulating  frequencies  and  “Ac”  and  denote 
their  magnitudes,  respectively.  The  signal  has  three  distinct  frequency  components,  namely,  fc,fc 
+fm,  and  fc  -fm.  The  spread  (see  figure  22b)  of  frequency  is  then  ±fm  around  the  carrier.  The 
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algorithm  for  detection  of  human  voice  consists  of  estimating  the  formant  (carrier  frequency)  and 
the  spread.  If  the  spread  is  above  some  threshold,  we  declare  it  is  a  human  voice.  Statistical 
analysis  is  performed  on  various  speech  signals  in  order  to  determine  the  threshold  value. 


Figure  21.  Sample  voice  data. 


Figure  22.  (a)  Portion  of  voice  signal  in  figure  16  and  (b)  its  spectrum. 

4.1.2  Detection  of  People  Using  Energy  in  Several  Bands 

It  is  known  (5)  that  the  human  voice  spans  50  Hz-20  kHz  frequency  range;  however,  most  of  the 
energy  is  concentrated  in  four  bands,  as  can  be  seen  in  figure  22b.  These  bands  are  50-250, 
251-500,  501-750,  and  751-1000  Hz.  The  energy  levels  in  these  bands  are  the  features  and 
designated  by  the  feature  vector  X=  ,  where  x,  is  the  energy  in  band  ‘i\  The  feature 

vectors  are  used  to  classify  whether  they  belong  to  human  voice  or  not  using  a  multivariate 
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Gaussian  (MYG)  classifier,  as  described  in  reference  7.  For  the  sake  of  continuity,  we  present  a 
short  description  of  the  MVG  classifier.  We  assume  the  energy  levels  in  each  band  are 
statistically  independent  and  have  the  Gaussian  distribution  given  by 

=  -  J.v.,/;l!»p{-X<-y, -M')T  r'<X‘  ’  (2) 

(2tt)  \Lj\ 

where  M,  and  £i  denote  the  mean  and  variance  respectively  and  T  denotes  the  transpose.  Then, 
the  likelihood  that  a  person  is  present  or  not  is  given  by 

N 

p(x  I  Hj )  =  Yl p{*i  I  Hj  )  p(Hj  \  j  =  {0, 1},  (3) 

i= 1 

where  Ho  and  H\  are  the  hypotheses  that  correspond  to  a  person  is  not  present  and  a  person  is 
present,  respectively.  Then,  the  posterior  probability  of  human  presence  is  given  by 

N 

Y\p(x,  \Hx)p(Hl) 

P{HX  |  X)  =  — - ^ - - - .  (4) 

n  i>ix'  \h,)p(h,)+  n  ”ix'  \Hl)p{Hl) 

i= 1  1=1 

Assuming  the  priors  p(Ho )  =p(H\)  =  0.5,  we  can  compute  the  posterior  probability  of  a  human 
present  given  X.  If  it  exceeds  a  particular  threshold  value,  we  declare  that  a  human  is  detected. 
The  algorithm  is  given  as  follows: 

Algorithm  1: 

1.  Let  s(t)  correspond  to  1-s  data 

2.  S  =fft(s )  is  the  Fast  Fourier  transform  of  the  signal  s(t). 

3.  Compute  the  energy  in  each  band.  These  energies  become  the  feature  vector 
X=  (xi  vX2,X3,x4J.  Compute  the  mean  and  variances  for  feature  x,  for  all  i. 

4.  Use  equations  2  and  4  with  appropriate  means  and  variances  for  noise  and  statistics 
collected  on  people  to  compute  the  posterior  probability  p{H\\X) 

5.  Use  the  posterior  probability  for  fusion  and  declare  that  a  person  is  detected  if p{H\\X)  > 

0.6. 

An  additional  component  of  the  acoustic  data  analysis  is  footstep  detection.  This  is  the  same  as 
footstep  detection  using  seismic  sensor  data  and  is  covered  in  the  next  section.  Figure  23  shows 
the  block  diagram  for  acoustic  data  analysis  for  personnel  detection. 
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Figure  23.  Acoustic  data  processing. 


4.2  Seismic  Data  Analysis 

The  main  purpose  of  seismic  sensor  is  to  detect  footfalls  of  humans  walking  within  the  receptive 
field  of  the  sensor.  There  is  a  considerable  amount  of  literature  ( 1-6)  in  footstep  detection. 
Traditionally,  they  focused  on  estimation  of  cadence.  However,  if  multiple  people  are  in  the 
vicinity  of  the  sensor  and  walking,  it  is  difficult  to  estimate  the  cadence  of  an  individual  person. 
Moreover,  if  there  are  animals  present,  it  is  difficult  to  differentiate  multiple  people  walking  and 
animals  walking  by  observing  the  footfalls.  However,  multiple  footfalls  superpose  one  another 
resulting  in  a  frequency  of  “c”  Hz  (where  “c”  is  the  cadence  of  a  person)  along  with  several 
harmonics  of  “c.”  So,  seismic  algorithms  would  look  for  harmonics  of  cadence  or  several  strong 
frequency  components  between  2  to  15  Hz.  The  seismic  algorithm  is  similar  to  that  of  algorithm 
1,  once  the  feature  set  is  determined.  For  the  feature  set,  we  first  compute  the  spectrum  of  the 
envelop  (i,  7)  of  the  seismic  signal  accumulated  for  a  period  of  6  s  (to  encompass  several 
footfalls).  Then,  the  feature  set  consists  of  amplitudes  of  the  frequency  bins  from  2  to 
15  Hz.  These  amplitudes  become  the  feature  set  X=  {xi,  xo,-  ■  ■  An }  •  Then,  algorithm  1  is  used  to 
estimate  the  posterior  probability  of  footsteps  present. 

As  mentioned  earlier,  the  algorithm  only  determines  whether  there  are  footsteps  or  not.  However, 
in  order  to  determine  the  presence  of  humans,  it  is  necessary  to  determine  whether  these 
footsteps  belong  to  a  human  or  an  animal.  For  this,  we  invariably  turn  to  acoustics.  If  there  is 
voice,  it  can  be  detected  and  identified  as  human  voice  based  on  the  formants  as  described  in 
section  4.1.  In  order  to  distinguish  between  people  and  animals,  we  analyze  the  sound  generated 
by  the  animals  walking.  When  the  hoofs  of  a  horse  hit  the  ground,  they  make  a  distinct  sound. 
Figure  24a  shows  the  signature  of  a  horse  walking  (for  a  period  of  6  s)  and  figure  24b  shows  the 
signature  with  the  noise  removed.  The  noise  removal  is  performed  using  empirical  mode 
decomposition  ( 4 ,  6)  of  the  original  signal  into  various  component  signals.  From  figure  24,  it  is 
clear  that  there  are  three  peaks  uniformly  distributed  in  each  time  interval  of  one  second.  This 
indicates  the  cadence  of  the  horse  to  be  approximately  2.8  to  3  Hz.  Since  the  cadence  of  a  person 
is  around  1.5  to  2  Hz,  one  can  infer  the  presence  of  animals. 
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Figure  24.  (a)  Hoof  signature  (b)  hoof  signature  after  noise  removal. 

In  this  section,  we  process  the  ultrasonic  data.  This  is  primarily  analyzed  to  distinguish  the 
characteristic  arm  and  leg  movements  by  observing  the  micro-Doppler  pertaining  to  these  limbs. 
Typical  micro-Doppler  frequencies  from  moving  portions  (arms,  legs,  chest,  etc.)  of  a  person 
walking  for  the  active  ultrasonic  sensor  are  shown  in  figure  25.  Ideally,  the  micro-Doppler 
signatures  from  humans  will  be  sufficiently  different  from  animals  to  enable  target  classification. 
However,  since  the  analysis  of  the  micro-Doppler  signatures  is  still  ongoing,  it  remains  to  be 
seen  if  this  capability  can  be  realized.  Here,  we  are  able  to  process  the  ultrasonic  data  to  count 
the  number  of  targets  in  the  vicinity  using  the  energy  content  in  the  micro-Doppler  return. 

Figure  26  shows  the  flow  chart  for  the  algorithm  used  in  counting  the  number  of  targets.  For 
processing  the  ultrasonic  data,  a  1-s  interval  of  the  sampled  data  is  used,  and  the  algorithm 
shown  in  figure  26  is  used  to  find  the  energy  in  each  band.  Then  a  sliding  window  is  used,  which 
covers  10,000  samples  (approximately  0.1  s),  and  next  set  of  data  is  obtained  and  processed. 


Figure  25.  Micro-Doppler  signature  of  a  person  walking  for  the  active  ultrasonic  sensor. 
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Figure  26.  Flow  chart  for  ultrasonic  data  processing. 


In  section  5,  we  describe  the  data  collection  scenarios  used  and  show  some  of  the  results  of 
various  algorithms  on  real  data  collected  at  southwest  border. 


5.  Data  Collection  and  Algorithm  Results 


In  order  to  collect  data  in  a  realistic  environment,  we  traveled  to  the  U.S.  southwest  border  and 
performed  the  experiments  on  routes  typically  used  by  smugglers.  We  selected  three  types  of 
common  travel  routes  used  by  illegal  immigrants  crossing  the  border: 

1 .  A  wash,  a  flashflood  river  bed  with  fine  grain  sand  interspersed  with  rocks 

2.  A  trail,  a  worn  path  through  the  shrubs  and  bushes  usually  heading  towards  a  pick-up 
destination 

3.  A  choke  point,  a  valley  between  two  mountains  or  large  hills 

During  the  data  collection  experiment,  we  used  several  sensor  modalities:  acoustic,  seismic,  PIR, 
magnetic,  E-field,  active  and  passive  ultrasonic,  IR  and  visible  imagers,  and  video  sensors.  Each 
sensor  suite  was  alternately  placed  along  the  selected  travel  path  about  50  m  apart.  Some  of  the 
scenarios  used  for  data  collection  experiment  included  (1)  a  single  person  walking  with  and 
without  a  back  pack,  (2)  two  people  walking,  (3)  multiple  people  walking,  (4)  one  person  leading 
an  animal,  (5)  two  people  leading  animals,  and  (6)  three  people  leading  animals  with  and  without 
payloads.  A  total  of  26  scenarios  with  various  combinations  of  people,  animals  and  payload  were 
run  and  data  were  collected  for  the  three  chosen  test  sites.  The  data  were  collected  over  a  period 
of  three  days,  at  a  different  site  and  different  environment  each  day.  At  times,  there  was  wind 
and  other  times  it  was  still  and  quiet. 
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5.1  Voice  Analysis  and  Algorithm  Results 

Acoustic  data  was  analyzed  for  the  presence  of  formants  pertaining  to  human  speech.  The 
algorithm  results  are  shown  in  figure  27a  for  a  segment  of  data  collected  during  the  experiment. 
Figure  27b  is  a  bar  chart  showing  the  detections  of  humans.  The  results  of  the  analysis  are 
verified  by  playing  back  the  recorded  audio  ground  truth  and  listening  for  the  human  voices.  The 
detection  at  time  period  “9  s”  is  clearly  a  false  alarm. 


Figure  27.  (a)  Acoustic  data  and  (b)  detections. 

5.2  Seismic  Data  Analysis  and  Algorithm  Results 

The  seismic  data  was  analyzed  for  footstep  detection,  as  described  in  section  4.2,  and  the  results 
are  shown  in  figure  28a.  In  figure  28b,  we  find  that  the  detection  algorithm  did  not  detect  initial 
footsteps  for  about  8  s  as  the  amplitudes  of  the  footsteps  are  very  weak.  Once  the  footsteps  are 
detected,  the  algorithm  continued  to  detect  until  the  footsteps  faded  away.  We  also  notice  there 
are  a  few  misdetections.  The  initial  lapse  is  due  to  the  fact  that  6  s  worth  of  data  is  needed  to 
analyze  the  foot  step  detection.  The  misdetections  could  be  due  to  the  training  data. 
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Figure  28.  (a)  Seismic  data  and  (b)  detections. 

5.3  Ultrasonic  Data  Analysis  for  Counting  Number  of  Targets 

The  ultrasonic  data  was  analyzed  for  counting  the  number  of  targets  using  the  algorithm  shown 
in  figure  26.  The  results  are  shown  in  figure  29.  Each  subfigure  in  figure  29  corresponds  to  one 
of  the  scenarios  enacted  during  the  data  collection  test.  These  figures  show  how  many  people  and 
animals  are  walking  as  analyzed  by  the  ultrasonic  sensor.  The  x-axis  of  each  subfigure 
corresponds  to  the  number  of  iterations  the  algorithm  uses;  for  each  iteration,  the  data  set  is  time 
shifted  by  0.1  s.  The  number  of  distinct  peaks  in  each  subfigure  shows  the  number  of  targets. 


Figure  29.  Results  of  ultrasonic  data  processing. 
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6.  Conclusion 


The  goal  of  the  experiment  was  to  collect  personnel,  animal,  and  vehicle  signatures  at  active 
southwestern  U.S.  border  crossing  locations  using  an  array  of  multimodal  sensors.  The  resulting 
data  set  allows  ARL  researchers  to  analyze  and  better  understand  target  phenomenology,  to 
provide  signatures  for  the  development  of  multi-sensor  fusion  algorithms,  and  to  enable  the 
creation  of  detection  and  classification  algorithms  (for  single  and  multi-sensor  systems)  and 
target  counting  algorithms.  An  added  benefit  of  the  research  will  be  the  determination  of  which 
sensors  are  best  suited  to  work  in  concert,  and  which  algorithms  and  sensors  maximize  the 
probability  of  correctly  classifying  humans  versus  animals  versus  vehicles  for  tomorrow’s 
unattended  ground  sensor  systems.  The  initial  research  findings  using  border  crossing 
multimodal  data  show  great  promise  in  the  development  of  sensor  fusion  algorithms  for  target 
discrimination,  and  likely  will  continue  to  be  a  focus  of  researchers  at  ARL  and  academia  in  the 
near  future. 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


ARDEC 

Armament  Research  and  Development  Center 

ARL 

U.S.  Army  Research  Laboratory 

B&K 

Bruel  &  Kjaer 

CW 

continuous-wave 

DAQ 

data  acquisition 

GPS 

global  positioning  system 

FOV 

field  of  view 

ISR 

Intelligence,  Surveillance,  and  Reconnaissance 

MVG 

multivariate  Gaussian 

NVESD 

Night  Vision  and  Electronic  Sensors  Directorate 

P/N 

part  number 

PIR 

passive  infrared 

TRSS 

Tactical  Remote  Sensor  System 

UDS 

Doppler  sensor 

UR 

microphone  receiver 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ADMNSTR 

ELEC  DEFNS  TECHL  INFO  CTR 
ATTN  DTIC  OCP 

8725  JOHN  J  KINGMAN  RD  STE  0944 
FT  BEL  VOIR  VA  22060-6218 

1  CD  OFC  OF  THE  SECY  OF  DEFNS 
ATTN  ODDRE  (R&AT) 

THE  PENTAGON 
WASHINGTON  DC  20301-3080 

1  US  ARMY  RSRCH  DEV  &  ENGRG 
CMND 

ARMAMENT  RSRCH  DEV  &  ENGRG 
CTR 

ARMAMENT  ENGRG  &  TECHNLGY 
CTR 

ATTN  AMSRD  AAR  AEF  T  J  MATTS 
BLDG  305 

APG  MD  21005-5001 

1  PM  TIMS  PROFILER  (MMS-P) 
AN/TMQ-52 
ATTN  B  GRIFFIES 
BUILDING  563 
FT  MONMOUTH  NJ  07703 

1  US  ARMY  INFO  SYS  ENGRG  CMND 
ATTN  AMSEL  IE  TD  A  RIVERA 
FT  HUACHUCA  AZ  85613-5300 

1  COMMANDER 

US  ARMY  RDECOM 

ATTN  AMSRD  AMR  W  C  MCCORKLE 

5400  FOWLER  RD 

REDSTONE  ARSENAL  AL  35898-5000 

1  US  GOVERNMENT  PRINT  OFF 

DEPOSITORY  RECEIVING  SECTION 
ATTN  MAIL  STOP  ID  AD  J  TATE 
732  NORTH  CAPITOL  ST  NW 
WASHINGTON  DC  20402 

1  US  ARMY  CERDEC 

ATTN:  ROBERT  FORESTA 
INTELLIGENCE  &  INFO  WARFARE  DIR 
BLDG  6003 
APG  MD  21005 


1  US  ARMY  RSRCH  LAB 

ATTN  RDRL  CIM  G  T  LANDFRIED 

BLDG  4600 

APG  MD  21005-5066 

25  US  ARMY  RSRCH  LAB 

ATTN  IMNE  ALC  HRR  MAIL  & 
RECORDS  MGMT 
ATTN  RDRL  CIM  L  TECHL  LIB 
ATTN  RDRL  CIM  P  TECHL  PUB 
ATTN  RDRL  SES  A 
R  DAMARLA  (5  COPIES) 

R  SARTAIN 
T  WALKER  (5  COPIES) 

N  SROUR 

J  HOUSER  (5  COPIES) 

K  BENNETT 
L  KAPLAN 
T  PHAM 

ATTN  RDRL  SES  E 
M  THIELKE 
ATTN  RDRL  SER  M 
JOHN  CLARK 
ADELPHI  MD  20783-1197 

3  US  ARMY  ARDEC 

ATTN  SHAFIK  QUORAISHEE 
ATTN  DAVID  GRASING 
ATTN  SACHI DESAI 
FUZE  PRECISION  ARMAMENT 
TECHNOLOGY  DIVISION 
PICATINNY  ARSENAL  NJ  07806 

1  US  ARMY  NIGHT  VISION  LAB 
ATTN  JOHN  HUTCHISON 
ROBOTICS  URBAN  &  GROUND 
SENSORS 

FORT  BEL  VOIR  VA  22060-5806 

1  UNIVERSITY  OF  MISSISSIPPI 
ATTN  JAMES  M.  SABATIER 
NATIONAL  CTR  FOR  PHYSICAL 
ACOUSTICS 
1  COLISEUM  DR. 

UNIVERSITY  MS  38677 
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NO.  OF 

COPIES  ORGANIZATION 

5  UNIVERSITY  OF  MEMPHIS 

ATTN  DR.  SRIKANT  KRISHNASWAMI 
CHARI 

ATTN  R.  KENNY  REYNOLDS 
ATTN  JEREMEY  BENJAMIN  BROWN 
ATTN  JASON  MICHAEL  BROOKS 
ATTN  FORREST  ALBERT  SMITH 
206  ENGINEERING  SCIENCES  BLVD 
MEMPHIS  TN  38152 

TOTAL:44  (1  ELEC,  1  CD,  42  HCS) 


34 


